1 Introduction to stochastic processes

2 Time series

Observations:

• \(y_t \in \mathbb{R}\hspace{0.08cm}\) is a realization of the random variable \(\hspace{0.08cm}\mathcal{Y}_t\) \(\\[0.35cm]\)

• A time series is a realization of a stochastic process. The time series is considered a result or trajectory of the stochastic process. \(\\[0.35cm]\)

• A time series can be defined as a vector of data points ordered in time. Where the data is equally spaced in time, namely, between each data point there is the same time space, such as a week, a month, a trimester, a quarter …

The process is characterized by the join probability distribution of the random variables \(\hspace{0.1cm} \mathcal{Y}_1 , \mathcal{Y}_2 ,..., \mathcal{Y}_k \hspace{0.1cm}\), namely, is characterized by the join density or probability function \(\hspace{0.08cm}f_{\hspace{0.08cm}\mathcal{Y}_1 , \mathcal{Y}_2 ,..., \mathcal{Y}_k}\)

This distribution is called finite-dimensional distribution of the process. We say that we know the probabilistic structure of the stochastic process when we know that join distribution, which determine the distribution of any subset of the variables and, in particular, the marginal distribution of each variable.

3 Mean function

Observations:

• An important particular case, due to its simplicity, arises when all the variables have the same mean and thus the mean function is a constant. The realizations of the process show no trend and we say that the process is stable in the mean. \(\\[0.35cm]\)

• If, on the contrary, the means change over time, the observations at different moments will reveal that change. \(\\[0.35cm]\)

• On many occasions we only have one realization of the stochastic process and we have to deduce from that whether the mean function of the process is, or is not, constant over time.

4 Variance function

Observations:

• We say that the process is stable in the variance if the variability is constant over time. \(\\[0.3cm]\)

• A process can be stable in the mean but not in the variance and vice versa.

5 Autocovariance function

The structure of linear dependence between random variables is represented by the covariance and correlation functions.

The autocovariances have dimensions, the squares of the series, thus it is not advisable to use them for comparing series measured in different units.

6 Autocorrelation function

It is interesting to notice the differences between conditional distributions and the marginal distributions.

The marginal distribution of \(\hspace{0.08cm}\mathcal{Y}_t\hspace{0.08cm}\) represents what we know about a variable, without knowing anything about its trajectory until time \(\hspace{0.05cm}t\hspace{0.05cm}\).

The conditional distribution of \(\hspace{0.08cm}\mathcal{Y}_t\hspace{0.08cm}\) given \(\hspace{0.08cm}\mathcal{Y}_{t-1}\hspace{0.05cm}\),…,\(\hspace{0.05cm}\mathcal{Y}_{t-r}\hspace{0.08cm}\) represents what we know about a variable when we know the k previous values of the process.

In time series conditional distributions are of greater interest than marginal ones because they define the predictions that we can make about the future knowing the past.

7 Time Series Descomposition

Time series decomposition is a process by which we separate a time series into its components: trend, seasonality and residuals.

• Trend represents the slow-moving changes in a time series. \(\\[0.25cm]\)

• Seasonality represent the seasonal pattern in the series. The cycles occur repeatedly over a fixed period of time. \(\\[0.25cm]\)

• Residuals represent the behavior that cannot be explained by the trend and seasonality components. \(\\[0.25cm]\)

from statsmodels.tsa.seasonal import seasonal_decompose, STL

Weekly_Time_Series_1

advanced_decomposition = STL(Weekly_Time_Series_1.IMPVENTA, period=4).fit()

fig, axs = plt.subplots(nrows=4, ncols=1, sharex=True)

plt.title("Time Series Decomposition",  fontsize = 16)

p1=sns.lineplot(advanced_decomposition.observed, color='red', ax=axs[0])
p2=sns.lineplot(advanced_decomposition.trend, color='red', ax=axs[1])
p3=sns.lineplot(advanced_decomposition.seasonal, color='red', ax=axs[2])
p4=sns.lineplot(advanced_decomposition.resid, color='red', ax=axs[3])

p1.set_ylabel('Observed')
p2.set_ylabel('trend')
p3.set_ylabel('seasonal')
p4.set_ylabel('resid')

p1.set_xticks(np.arange(0 , len(Weekly_Time_Series_1) , 10))

plt.setp(p1.get_xticklabels(), rotation=90)

fig.savefig('p6.jpg', format='jpg', dpi=1200)

plt.show()

fig, ax = plt.subplots()

p1=sns.lineplot(x="Semana-Mes-Año", y="IMPVENTA", data=Weekly_Time_Series_1 , color='red')
p2=sns.lineplot(advanced_decomposition.trend, color='blue', linestyle='-', label='Trend')

p1.set_xticks(np.arange(0 , len(Weekly_Time_Series_1) , 10))

plt.setp(p1.get_xticklabels(), rotation=90)

plt.title("Trend and Time Series",  fontsize = 17)

fig.savefig('p7.jpg', format='jpg', dpi=1200)

plt.show()

8 Regression Problem vs Time Series Forecasting

You probably have encountered regression problems where you must predict some continuous target given a certain set of features. At first glance, time series forecasting seems like a typical regression problem: we have some historical data, and we wish to build a mathematical expression that will express future values as a function of past values. However, there are some key differences between time series forecasting and regression for time-independent scenarios that deserve to be addressed before we look at our very first forecasting technique.

• Time series have an order:

  The first concept to keep in mind is that time series have an order, and we cannot change that order when modeling. In time series forecasting, we express future values as a function of past values. Therefore, we must keep the data in order, so as to not violate this relationship.

  Other regression tasks in machine learning often do not have an order. For example, if you are tasked to predict revenue based on ad spend, it does not matter when a certain amount was spent on ads. Instead, you simply want to relate the amount of ad spend to the revenue. In fact, you might even randomly shuffle the data to make your model more robust. Here the regression task is to simply derive a function such that given an amount on ad spend, an estimate of revenue is returned. On the other hand, time series are indexed by time, and that order must be kept. \(\\[1.55cm]\)

• Time series sometimes do not have features:

  It is possible to forecast time series without the use of features other than the time series itself. As data scientists, we are used to having datasets with many columns, each representing a potential predictor for our target. For example, consider the task of predicting revenue based on ad spend, where the revenue is the target variable. As features, we could have the amount spent on Google ads, Facebook ads, and television ads. Using these three features, we would build a regression model to estimate revenue. However, with time series, it is quite common to be given a simple dataset with a time column and a value at that point in time. Without any other features, we must learn ways of using past values of the time series to forecast future values. This is when the moving average model or autoregressive model come into play, as they are ways to express future values as a function of past values

9 Time Series Forecasting

Given a stochastic process \(\hspace{0.15cm} \mathcal{Y} \hspace{0.1cm}=\hspace{0.1cm} \hspace{0.1cm}\bigl( \hspace{0.06cm} \mathcal{Y}_1 , \mathcal{Y}_2 ,..., \mathcal{Y}_n \hspace{0.06cm}\bigl) \hspace{0.1cm}\) .

Observations:

• We are assuming that the available data is \(\hspace{0.1cm}Y_t = \left( y_1, y_2, ...,y_n \right)^t \hspace{0.1cm}\). \(\\[0.35cm]\)

• In many cases \(\hspace{0.1cm}y_{n+k}\hspace{0.1cm}\) is the value of the process in a future period. So, by definition, we don’t know \(\hspace{0.1cm}y_{n+k}\hspace{0.12cm}\), for \(\hspace{0.07cm}k=1,2,3,\dots\).

10 Stationary processes

11 White Noise Process

Given a stochastic process \(\hspace{0.15cm} \mathcal{Y} \hspace{0.08cm}=\hspace{0.08cm} \hspace{0.1cm}\bigl( \hspace{0.06cm} \mathcal{Y}_1 , \mathcal{Y}_2 ,..., \mathcal{Y}_n \hspace{0.06cm}\bigl) \hspace{0.1cm}.\) \(\\[0.25cm]\)

The process \(\hspace{0.08cm}\mathcal{Y}\hspace{0.08cm}\) is a white noise process if: \(\\[0.35cm]\)

• \(E[\mathcal{Y}_t] \hspace{0.08cm}=\hspace{0.08cm} 0 \hspace{0.3cm}, \hspace{0.3cm}\forall\hspace{0.08cm} t=1,\dots , n\\\)

• \(Var(\mathcal{Y}_t)=\sigma^2 \hspace{0.3cm}, \hspace{0.3cm}\forall\hspace{0.08cm} t=1,\dots , n\\\)

• \(\gamma_{t,t+h} \hspace{0.08cm}= \hspace{0.08cm}Cov(\mathcal{Y}_t, \mathcal{Y}_{t+h})\hspace{0.08cm}=\hspace{0.08cm}0 \hspace{0.3cm}, \hspace{0.3cm}\forall\hspace{0.08cm} t=1,\dots , n\\\)

Observation:

A white noise process is a weak stationarity process.

12 First-Order Autoregresive Process: AR(1) process

Given a stochastic process \(\hspace{0.15cm} \mathcal{Y} \hspace{0.08cm}=\hspace{0.08cm} \hspace{0.1cm}\bigl( \hspace{0.06cm} \mathcal{Y}_1 , \mathcal{Y}_2 ,..., \mathcal{Y}_n \hspace{0.06cm}\bigl) \hspace{0.1cm}.\) \(\\[0.25cm]\)

The first-order autoregressive process \(\hspace{0.08cm}AR(1)\hspace{0.08cm}\) for the process \(\hspace{0.08cm}\mathcal{Y}\hspace{0.08cm}\) is defined as: \(\\[0.35cm]\)

\[\mathcal{Y}_t\hspace{0.08cm}=\hspace{0.08cm}\phi_0 \hspace{0.08cm}+\hspace{0.08cm} \phi_1 \cdot \mathcal{Y}_{t-1} \hspace{0.08cm}+\hspace{0.08cm} \varepsilon_t \hspace{0.4cm} , \hspace{0.4cm} t = 2,3,\dots , n\]

Where:

• \(\varepsilon_t \hspace{0.08cm}\sim \hspace{0.08cm}N(0,\sigma^2) \hspace{0.3cm},\hspace{0.3cm} \forall\hspace{0.08cm} t = 2,3,\dots , n\\\)

• \(Cov(\varepsilon_t \hspace{0.08cm},\hspace{0.08cm} \varepsilon_{t+h})=0 \hspace{0.3cm},\hspace{0.3cm} \forall\hspace{0.08cm} t = 2,3,\dots \hspace{0.15cm},\hspace{0.15cm} \forall\hspace{0.08cm} h = \pm 1,\pm 2,\dots\\\)

• \(Cov(\mathcal{Y}_{t+h}\hspace{0.08cm},\hspace{0.08cm} \varepsilon_t) \hspace{0.08cm}=\hspace{0.08cm} 0 \hspace{0.3cm},\hspace{0.3cm} \forall\hspace{0.08cm} t = 2,3,\dots \hspace{0.15cm},\hspace{0.15cm} \forall\hspace{0.08cm} h = 0,\pm 1,\pm 2,\dots\\\)

  First two assumptions means that \(\hspace{0.08cm}(\hspace{0.03cm}\varepsilon_t \hspace{0.1cm}:\hspace{0.1cm} t = 2,3,\dots, n \hspace{0.03cm})\hspace{0.1cm}\) is a normal white noise proccess.

13 First-Order Autoregresive Model: AR(1) model

In practice, we have a time series \(\hspace{0.07cm}Y_t = \left( y_1, y_2, ...,y_n \right)^t \hspace{0.1cm}\) associated to the stochastic process \(\hspace{0.07cm}\mathcal{Y}\hspace{0.07cm}\). \(\hspace{0.07cm}\)

Where \(\hspace{0.05cm}y_t\hspace{0.07cm}\) is the value of the process \(\hspace{0.07cm}\mathcal{Y}\hspace{0.07cm}\) at period \(\hspace{0.07cm}t\). \(\\[0.4cm]\)

Our main goal is to predict the value of the process at future periods. For that, we will use the observed values of the process in the time series, namely, the available data. \(\\[0.3cm]\)

In this sense, we can use \(\hspace{0.07cm}AR(1)\hspace{0.07cm}\) process as a model to predict (forecast) time series.

Let be \(\hspace{0.09cm}\widehat{y}_t\hspace{0.09cm}\) the model prediction for \(\hspace{0.09cm}y_t\hspace{0.09cm}\) observation of the process.

We can use an \(\hspace{0.07cm}AR(1)\hspace{0.07cm}\) to estimate past values of the process \(\hspace{0.09cm}\mathcal{Y}\hspace{0.09cm}\) or to forecast future values of it.

14 AR(1) model in Python

First of all, we import some of the libraries we will be using throughout this article:

import pandas as pd
import numpy as np

import matplotlib.pyplot as plt
import seaborn as sns

sns.set_style("darkgrid")

We load a time series data-set, in this case the minimum periodicity of the time series is daily:

Time_Series_1 = pd.read_csv('Time_Series_1.csv')

We are going to expose some preprocessing time series task, not many but important.

When we have a time series, usually we will have a date column, in this data-set called Fecha.

When we have identified the date column we must convert it to date-time format. We can do it using Pandas as follows:

Time_Series_1['Fecha'] = pd.to_datetime(Time_Series_1['Fecha'])

Other important preprocessing task when we have a time series data-set is to create new columns with the day, week, month, quarter, and year associated to each observation.

We can carry out these tasks with Pandas:

Time_Series_1['Dia'] = Time_Series_1['Fecha'].dt.day

Time_Series_1['Semana'] = Time_Series_1['Fecha'].dt.week

Time_Series_1['Mes'] = Time_Series_1['Fecha'].dt.month

Time_Series_1['Trimestre'] = Time_Series_1['Fecha'].dt.quarter

Time_Series_1['Año'] = Time_Series_1['Fecha'].dt.year

We select the specific columns we will use throughout this article. The response variable will be the Ventas column, and the rest are columns related with the observation date.

Time_Series_1 = Time_Series_1.loc[: , ['Fecha', 'Dia', 'Semana', 'Mes', 'Trimestre', 'Año', 'Ventas']]

Time_Series_1

Los modelos AR (autoregresivos) son un tipo de modelo de series temporales en el que una variable se modela como una función lineal de sus valores pasados. El modelo AR(p) especifica que el valor de la variable en el tiempo t depende linealmente de los p valores anteriores de la variable, es decir,

\[y_t \hspace{0.1cm}=\hspace{0.1cm} \phi_0 \hspace{0.05cm}+\hspace{0.05cm} \sum_{i=1}^p \phi_i \cdot y_{t-i} \hspace{0.05cm}+\hspace{0.05cm} \varepsilon_t \hspace{0.1cm}=\hspace{0.1cm} \phi \hspace{0.05cm}+\hspace{0.05cm} \phi_1 \cdot y_{t-1} \hspace{0.05cm}+\hspace{0.05cm} \phi_2 \cdot y_{t-2}\hspace{0.05cm}+\hspace{0.05cm} \cdots \hspace{0.05cm}+\hspace{0.05cm} \phi_p \cdot y_{t-p} \hspace{0.05cm}+\hspace{0.05cm} \varepsilon_t\]

donde:

• Se asume que \(\hspace{0.05cm}\varepsilon_t\hspace{0.05cm}\) es una v.a. con \(\hspace{0.05cm}E[\varepsilon_t]=0\hspace{0.07cm}\) y \(\hspace{0.07cm}Var(\varepsilon_t)=\sigma^2\hspace{0.05cm}\) \(\\[0.3cm]\)

• \(y_t\hspace{0.05cm}\) es el valor de la variable en el tiempo \(\hspace{0.05cm}t\). \(\\[0.3cm]\)

• \(\phi_0, \phi_1, \ldots, \phi_p\hspace{0.05cm}\) son parametros a estimar. Son los coeficientes asociados a cada uno de los \(\hspace{0.05cm}p\hspace{0.05cm}\) retardos del proceso.\(\\[0.3cm]\)

Expresión AR(p) en funcion del operador de retardos:

El modelo \(\hspace{0.07cm}AR(p)\hspace{0.07cm}\) puede expresarse en funcion del operador de retardos \(\hspace{0.07cm}B\hspace{0.07cm}\) como sigue:

\[\phi_p(B) \cdot y_t \hspace{0.1cm} = \hspace{0.1cm} \left( 1- \phi_1 \cdot B - \phi_2 \cdot B^2 - \dots - \phi_p \cdot B^p \right)\cdot y_t \hspace{0.1cm} = \hspace{0.1cm} \varepsilon_t + \phi_0\]

donde:

\[\phi_p(B) \hspace{0.1cm}=\hspace{0.1cm} \left(1- \phi_1 \cdot B - \phi_2 \cdot B^2 - \dots - \phi_p \cdot B^p \right)\]

Estimaciones del pasado:

Suponiendo que la informacion disponible para realizar la estimación del modelo (estimación de parametros) es \(\hspace{0.08cm}Y_t = (y_1,\dots, y_n)^t\hspace{0.08cm}\), en el modelo \(\hspace{0.07cm} AR(p) \hspace{0.07cm}\) tenemos las siguientes ecuaciones de estimación de los valores pasados del proceso: \(\\[0.25cm]\)

\[\widehat{y}_{t} \hspace{0.1cm}=\hspace{0.1cm} c \hspace{0.05cm}+\hspace{0.05cm} \sum_{h=1}^p \hspace{0.05cm}\widehat{\phi}_h \cdot y_{\hspace{0.05cm} t - h} \hspace{0.1cm}=\hspace{0.1cm} c \hspace{0.07cm}+\hspace{0.07cm} \widehat{\phi}_1 \cdot y_{\hspace{0.05cm}t-1} \hspace{0.07cm}+\hspace{0.07cm} \widehat{\phi}_2\cdot y_{\hspace{0.05cm}t-2} \hspace{0.07cm}+\hspace{0.07cm} \cdots \hspace{0.07cm}+\hspace{0.07cm} \widehat{\phi}_p \cdot y_{\hspace{0.05cm}t-p} \hspace{0.25cm} , \hspace{0.25cm} t=1,\dots , n\]

Predicciones del futuro:

Suponiendo que la informacion disponible para realizar la estimación del modelo (estimación de parametros) es \(\hspace{0.08cm}Y_t = (y_1,\dots, y_n)^t\hspace{0.08cm}\), en un modelo \(\hspace{0.07cm}AR(p)\hspace{0.07cm}\) tenemos las siguientes ecuaciones de estimación de los valores pasados del proceso: \(\\[0.25cm]\)

\[\widehat{y}_{n+1} \hspace{0.1cm}=\hspace{0.1cm} \phi_0 \hspace{0.05cm}+\hspace{0.05cm} \sum_{h=1}^p \hspace{0.05cm}\widehat{\phi}_h \cdot y_{(n+1) - h} \hspace{0.1cm}=\hspace{0.1cm} \phi_0 \hspace{0.07cm}+\hspace{0.07cm} \widehat{\phi}_1 \cdot y_{(n+1)-1} \hspace{0.07cm}+\hspace{0.07cm} \widehat{\phi}_2\cdot y_{(n+1)-2} \hspace{0.07cm}+\hspace{0.07cm} \cdots \hspace{0.07cm}+\hspace{0.07cm} \widehat{\phi}_p \cdot y_{(n+1)-p} = \\ \hspace{3cm}=\hspace{0.1cm} \phi_0 \hspace{0.07cm}+\hspace{0.07cm} \widehat{\phi}_1 \cdot y_{n} \hspace{0.07cm}+\hspace{0.07cm} \widehat{\phi}_2\cdot y_{n-1} \hspace{0.07cm}+\hspace{0.07cm} \widehat{\phi}_3\cdot y_{n-2} \hspace{0.07cm}+\hspace{0.07cm} \cdots \hspace{0.07cm}+\hspace{0.07cm} \widehat{\phi}_p \cdot y_{n-(p-1)} \\[0.25cm]\]

\[\widehat{y}_{n+2} \hspace{0.1cm}=\hspace{0.1cm} \phi_0 \hspace{0.07cm}+\hspace{0.07cm} \widehat{\phi}_1 \cdot \widehat{y}_{(n+2)-1} \hspace{0.07cm}+\hspace{0.07cm} \sum_{h=2}^p \hspace{0.05cm} \widehat{\phi}_h \cdot y_{(n+2) - h} \hspace{0.1cm}=\hspace{0.1cm} \phi_0 \hspace{0.05cm}+\hspace{0.05cm} \widehat{\phi}_1 \cdot \widehat{y}_{n+1} \hspace{0.05cm}+\hspace{0.05cm} \widehat{\phi}_2 \cdot y_{n} \hspace{0.05cm}+\hspace{0.05cm} \widehat{\phi}_3\cdot y_{n-1} \hspace{0.05cm}+\hspace{0.05cm} \cdots \hspace{0.05cm}+\hspace{0.05cm} \widehat{\phi}_p \cdot y_{n-(p-2)} \\[0.3cm]\]

\[\widehat{y}_{n+3} \hspace{0.1cm}=\hspace{0.1cm} \phi_0 \hspace{0.07cm}+\hspace{0.07cm} \widehat{\phi}_1 \cdot \widehat{y}_{(n+3)-1} \hspace{0.07cm}+\hspace{0.07cm} \widehat{\phi}_2 \cdot \widehat{y}_{(n+3)-2} \hspace{0.07cm}+\hspace{0.07cm} \sum_{h=3}^p \hspace{0.05cm} \widehat{\phi}_h \cdot y_{(n+3) - h} \hspace{0.1cm}=\hspace{0.1cm} \phi_0 \hspace{0.05cm}+\hspace{0.05cm} \widehat{\phi}_1 \cdot \widehat{y}_{n+2} \hspace{0.05cm}+\hspace{0.05cm} \widehat{\phi}_2 \cdot \widehat{y}_{n+1} \hspace{0.05cm}+\hspace{0.05cm} \widehat{\phi}_3 \cdot {y}_{n} \hspace{0.05cm}+\hspace{0.05cm} \widehat{\phi}_3\cdot y_{n-1} \hspace{0.05cm}+\hspace{0.05cm} \cdots \hspace{0.05cm}+\hspace{0.05cm} \widehat{\phi}_p \cdot y_{n-(p-2)} \\[0.5cm]\]

En general, para \(\hspace{0.1cm} k=1,2,3,\dots \\\)

\[\widehat{y}_{n+k} \hspace{0.1cm}=\hspace{0.1cm} \phi_0 \hspace{0.07cm}+\hspace{0.07cm} \sum_{h=1}^{k-1} \hspace{0.05cm} \widehat{\phi}_h \cdot \widehat{y}_{(n+k)-h} \hspace{0.07cm}+\hspace{0.07cm} \sum_{h=k}^p \hspace{0.05cm} \widehat{\phi}_h \cdot y_{(n+k) - h} \hspace{0.1cm}=\hspace{0.1cm} \phi_0 \hspace{0.05cm}+\hspace{0.05cm} \widehat{\phi}_1 \cdot \widehat{y}_{n+k-1} \hspace{0.05cm}+\hspace{0.05cm} \widehat{\phi}_2 \cdot \widehat{y}_{n+k-2} \hspace{0.05cm}+\hspace{0.05cm} \dots\hspace{0.05cm}+\hspace{0.05cm} \widehat{\phi}_{k-1} \cdot \widehat{y}_{n+k-(k-1) = n+1} \hspace{0.05cm}+\hspace{0.05cm} \widehat{\phi}_k \cdot \widehat{y}_{n+k-k = n} \hspace{0.05cm}+\hspace{0.05cm} \widehat{\phi}_3\cdot y_{n+k-(k+1)=n-1} \hspace{0.05cm}+\hspace{0.05cm} \widehat{\phi}_4\cdot y_{n+k-(k+2)=n-2} \hspace{0.05cm}+\hspace{0.05cm}\cdots \hspace{0.05cm}+\hspace{0.05cm} \widehat{\phi}_p \cdot y_{n+k-(k+p-2)} \\[0.3cm]\]

Estimación del modelo por máxima verosimilitud:

Para un modelo \(\hspace{0.05cm}AR(p)\hspace{0.05cm}\) con término de error \(\hspace{0.07cm}\varepsilon_t\hspace{0.07cm}\) Gaussiano, es decir, \(\hspace{0.07cm}\varepsilon_t \sim N(0,\sigma^2)\hspace{0.07cm}\) , se asume la siguiente relación para cada variable del proceso:

\[\mathcal{Y}_t \hspace{0.1cm}=\hspace{0.1cm} \phi_0 + \sum_{h=1}^p \phi_h \cdot y_{t-h} + \varepsilon_t\]

Como el termino de error es Gaussiano, tal y como se especificó antes, se deduce que:

\[\mathcal{Y}_t \sim N\left( \hspace{0.07cm} \phi_0 + \sum_{h=1}^p \phi_h \cdot y_{t-h} \hspace{0.07cm} ,\hspace{0.07cm} \sigma^2 \hspace{0.07cm} \right)\]

para cada \(\hspace{0.07cm}t=1,\dots , n\)

Por tanto, la función de verosimilitud del modelo \(\hspace{0.07cm}AR(p)\hspace{0.07cm}\) es:

\[\mathcal{L}(\phi_0,\phi_1,\dots, \phi_p, \sigma^2) \hspace{0.07cm}=\hspace{0.07cm} f_{\mathcal{Y}_1,...,\mathcal{Y}_n}(y_{p+1},y_{p+2},...,y_n) \hspace{0.07cm}=\hspace{0.07cm} \prod_{t=p+1}^n f_{\mathcal{Y}_t}(y_t) \hspace{0.07cm}=\hspace{0.07cm} \prod_{t=p+1}^n \frac{1}{\sqrt{2\pi\sigma^2}} \cdot \exp\left(-\frac{(y_t - \phi_0 + \sum_{i=1}^p \phi_i y_{t-i})^2}{2\sigma^2}\right)\]

La función de log-verosimilitud es:

\[ ln\left( \hspace{0.07cm} \mathcal{L}(\phi_0,\phi_1,\dots, \phi_p, \sigma^2)\hspace{0.07cm} \right) \hspace{0.07cm}=\hspace{0.07cm} -\frac{n-p}{2} \cdot \ln(2\pi) - \frac{n-p}{2}\cdot \ln(\sigma^2) - \frac{1}{2\sigma^2}\cdot \sum_{t=p+1}^n \left( y_t - \sum_{h=1}^p \phi_h y_{t-h} \right)^2 \]

La estimación de los parametros del modelo \(\hspace{0.07cm} AR(p)\hspace{0.07cm}\) se realiza como sigue: \(\\[0.3cm]\)

\[\widehat{\phi}_0, \widehat{\phi}_1, \dots , \widehat{\phi}_p , \widehat{\sigma}^2 \hspace{0.07cm} =\hspace{0.07cm} arg \hspace{0.07cm} \underset{\phi_0,\dots , \phi_p, \sigma^2}{Max} ln\left( \hspace{0.07cm} \mathcal{L}(\phi_0,\phi_1,\dots, \phi_p, \sigma^2)\hspace{0.07cm} \right)\]

Este problema se puede resolver a través de métodos numéricos como el método de Newton-Raphson o el método de gradiente descendente.

15 Second-Order Autoregressive process: AR(2) process

Condicion de estacionariedad debil, demostracion:

Tienes razón. La demostración completa es un poco más complicada que lo que he descrito antes. Te la muestro a continuación.

En general, la condición suficiente para que un proceso \(AR(2)\) sea débilmente estacionario es que las raíces de la ecuación característica

\[\phi(z) = 1 - \phi_1 z - \phi_2 z^2 = 0\]

estén fuera del círculo unitario en el plano complejo, es decir, que \(|\alpha_i| > 1\) para \(i=1,2\), donde \(\alpha_1\) y \(\alpha_2\) son las raíces de \(\phi(z)=0\).

Para ver por qué esto es cierto, primero escribimos el proceso \(AR(2)\) como

\[Y_t = \phi_1 Y_{t-1} + \phi_2 Y_{t-2} + \epsilon_t,\]

donde \(\epsilon_t\) es un ruido blanco con media cero y varianza \(\sigma^2\). Tomando la transformada de Laplace, obtenemos

\[\phi(z) \mathcal{L}(Y_t) = \mathcal{L}(\epsilon_t),\]

donde \(\mathcal{L}\) denota la transformada de Laplace. Resolviendo para \(\mathcal{L}(Y_t)\), tenemos

\[\mathcal{L}(Y_t) = \frac{\mathcal{L}(\epsilon_t)}{\phi(z)}.\]

Ahora, para que el proceso \(AR(2)\) sea estacionario, es necesario que \(\mathcal{L}(Y_t)\) exista para todo \(t\). Esto es equivalente a que el denominador \(\phi(z)\) no tenga raíces en el círculo unitario, porque las raíces en el círculo unitario generan polos de orden uno en el origen de la transformada de Laplace, lo que implica que la serie de Fourier correspondiente no converge. Por lo tanto, si \(\phi(z)\) tiene raíces en el círculo unitario, el proceso \(AR(2)\) no puede ser estacionario.

Por otro lado, si las raíces de \(\phi(z)\) están fuera del círculo unitario, entonces podemos expresar \(\phi(z)\) como

\[\phi(z) = (\alpha_1 - z)(\alpha_2 - z),\]

donde \(|\alpha_i| > 1\). Entonces, podemos expandir \(\mathcal{L}(Y_t)\) en fracciones parciales como

\[\mathcal{L}(Y_t) = \frac{c_1}{z-\alpha_1} + \frac{c_2}{z-\alpha_2},\]

donde \(c_1\) y \(c_2\) son constantes que dependen de los valores iniciales \(Y_0\) y \(Y_1\). Tomando la transformada inversa de Laplace, obtenemos

\[Y_t = c_1 \alpha_1^t + c_2 \alpha_2^t.\]

Como \(|\alpha_i|>1\), podemos ver que \(\alpha_i^t\) converge a cero cuando \(t\) tiende a infinito. Por lo tanto, \(Y_t\) converge a una constante cuando \(t\) tiende a infinito, lo que implica que el proceso \(AR(2)\) es débilmente estacionario.

En resumen, para que un proceso AR(2) sea débilmente estacionario, es necesario que las raíces del polinomio característico estén dentro del círculo unitario. Esto se debe a que las raíces del polinomio característico están relacionadas con los coeficientes del modelo AR(2) y, por lo tanto, con las propiedades estadísticas del proceso. Cuando las raíces están dentro del círculo unitario, los coeficientes están acotados y el proceso AR(2) es estacionario en media, varianza y covarianza.

Cabe señalar que la condición de que las raíces estén dentro del círculo unitario no es suficiente para garantizar la estacionariedad fuerte del proceso AR(2), que implica la invariancia del proceso bajo desplazamientos en el tiempo. La estacionariedad fuerte requiere que se cumplan algunas condiciones adicionales, como la existencia de ciertos momentos y la ergodicidad del proceso.

16 AR(p)

n primer lugar, escribimos el proceso \(AR(p)\) como

\[Y_t = \sum_{i=1}^p \phi_i Y_{t-i} + \epsilon_t,\]

donde \(\epsilon_t\) es un ruido blanco con media cero y varianza \(\sigma^2\). Tomando la transformada de Laplace, obtenemos

\[\phi(z) \mathcal{L}(Y_t) = \mathcal{L}(\epsilon_t),\]

donde \(\mathcal{L}\) denota la transformada de Laplace. Resolviendo para \(\mathcal{L}(Y_t)\), tenemos

\[\mathcal{L}(Y_t) = \frac{\mathcal{L}(\epsilon_t)}{\phi(z)}.\]

Ahora, para que el proceso \(AR(p)\) sea estacionario, es necesario que \(\mathcal{L}(Y_t)\) exista para todo \(t\). Esto es equivalente a que el denominador \(\phi(z)\) no tenga raíces en el círculo unitario, porque las raíces en el círculo unitario generan polos de orden uno en el origen de la transformada de Laplace, lo que implica que la serie de Fourier correspondiente no converge. Por lo tanto, si \(\phi(z)\) tiene raíces en el círculo unitario, el proceso \(AR(p)\) no puede ser estacionario.

Por otro lado, si las raíces de \(\phi(z)\) están fuera del círculo unitario, entonces podemos expresar \(\phi(z)\) como

\[\phi(z) = (\alpha_1 - z)(\alpha_2 - z) \cdots (\alpha_p - z),\]

donde \(|\alpha_i| > 1\). Entonces, podemos expandir \(\mathcal{L}(Y_t)\) en fracciones parciales como

\[\mathcal{L}(Y_t) = \frac{c_1}{z-\alpha_1} + \frac{c_2}{z-\alpha_2} + \cdots + \frac{c_p}{z-\alpha_p},\]

donde \(c_1,\dots,c_p\) son constantes que dependen de los valores iniciales \(Y_0,\dots,Y_{p-1}\). Tomando la transformada inversa de Laplace, obtenemos

\[Y_t = c_1 \alpha_1^t + c_2 \alpha_2^t + \cdots + c_p \alpha_p^t.\]

Como \(|\alpha_i|>1\), podemos ver que \(\alpha_i^t\) converge a cero cuando \(t\) tiende a infinito. Por lo tanto, \(Y_t\) converge a una constante cuando \(t\) tiende a infinito, lo que implica que el proceso \(AR(p)\) es débilmente estacionario.

En resumen, las raíces de la ecuación característica de un proceso \(AR(p)\) deben estar fuera del círculo unitario en el plano complejo para que el proceso sea débilmente estacionario.